Improved Distributed Principal Component Analysis

We study the distributed computing setting in which there are multiple servers,each holding a set of points, who wish to compute functions on the union of theirpoint sets. A key task in this setting is Principal Component Analysis (PCA), inwhich the servers would like to compute a low dimensional subspace capturing asmuch of the variance of the union of their point sets as possible. Given a proce-dure for approximate PCA, one can use it to approximately solve problems suchas k-means clustering and low rank approximation. The essential properties of anapproximate distributed PCA algorithm are its communication cost and computa-tional efficiency for a given desired accuracy in downstream applications. We givenew algorithms and analyses for distributed PCA which lead to improved com-munication and computational costs for k-means clustering and related problems.Our empirical study on real world data shows a speedup of orders of magnitude,preserving communication with only a negligible degradation in solution quality.Some of these techniques we develop, such as a general transformation from aconstant success probability subspace embedding to a high success probabilitysubspace embedding with a dimension and sparsity independent of the successprobability, may be of independent interest.